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Abstract 


Although about 40% of high school graduates who take the ACT® test express interest in pursuing a 
career in a science, technology, engineering, and mathematics (STEM) field, the percentage of first- 
year students in college who declare a STEM major is substantially lower. The pool of prospective 
STEM workers shrinks further as the majority of STEM majors do not earn a STEM degree. A lack of 
academic preparation in science and mathematics has been offered as one explanation for the leaky 
STEM pipeline. The purpose of this research was to develop STEM readiness benchmarks to provide 
prospective students more tailored information on the level of knowledge and skills needed to have 
a reasonable chance of success in first-year STEM courses. The research had three components. 

Study 1 identified fhe mathematics and science courses that STEM majors take most often in the 
first year of college. In mathemafics, fhe mosf prevalent course was Calculus. In science, multiple 
courses were identified as typically taken by STEM majors: Biology, Chemistry, Engineering, and 
Physics. 

Study 2 derived empirically based STEM readiness benchmarks in mathematics and science by 
estimating the ACT Mathematics and Science test scores associated with a 50% probability of 
earning a grade of a B or higher in the identified STEM courses. Specifically, the median ACT 
Mathematics score associated with a 50% probability of earning a B or higher grade in Calculus is 
27. The median ACT Science score associated with a 50% probability of earning a B or higher grade 
in Chemistry, Biology, Physics, or Engineering is 25. 

Study 3 validated the STEM readiness benchmarks on more distal indicators of success. Results 
demonstrated that STEM majors who met the STEM readiness benchmarks were more likely to earn 
a cumulative grade point average of 3.0 or higher, persist in a STEM major, and earn a STEM-related 
bachelor’s degree. Providing STEM readiness information to prospective students may help facilitate 
the transition to college by aligning students’ expectations with course demands. 


Introduction 


The ability of the United States to maintain a supply of science, technology, engineering, and 
mathematics (STEM) graduates sufficient to retain its competitiveness in the global economy 
has come under scrutiny (Langdon, McKittick, Beede, Khan, & Dorns, 2014; National Academy of 
Sciences, National Academy of Engineering, & Institute of Medicine, 2005; Obama, 2009; U.S. 
Department of Labor, 2007). After World War II, the United States led the world in educational 
achievement, but today the situation is quite different, according to a recent report by the National 
Science Board (NSB; 2014). Whereas in 2000 the United States produced nearly twice as many 
graduates with degrees in the physical or biological sciences as did China, in 2010 China produced 
more than twice as many graduates in these fields as did the United States. In 2010, China, India, 
Russia, Ukraine, and Japan all produced more engineering graduates than did the United States, and 
both South Korea and Mexico each produced nearly as many engineers as did the United States. 
Overall, of the 5.5 million science and engineering degrees awarded throughout the world in 201 0, 
2.5 million were awarded in Asia, 1.5 million were awarded in Europe, and 0.5 million were awarded 
in the United States. 

Given the importance of the STEM workforce in the modern global economy (Langdon, et al., 2014; 
U.S. Department of Labor, 2007), there have been calls for producing one million more STEM majors 
over the next decade (Executive Office of the President, President's Council of Advisors on Science 
and Technology, 2012). However, research indicates that students leave the STEM pipeline at various 
transition points along their education career. For example, recent findings indicate that about 
40% of high school graduates who take the ACT® test express interest in majoring in STEM (ACT, 
2014b). However, the percentage of students who actually declare a STEM major upon enrollment is 
substantially lower: in the 1995-1996 academic year, only 17% of undergraduates declared a STEM 
major in the first year (Chen, 2009). Based on the same cohort of students, results indicate that 
only a small percentage of students move into STEM after the first year, with an additional 6% of 
undergraduates declaring a STEM major after their first year (Chen, 2009). The pool of prospective 
STEM workers continues to shrink as the majority of STEM majors do not earn degrees in STEM 
fields. Only 37% of first-year STEM majors earned a degree or certificate within six years. Data on 
a more recent cohort paints a similar picture (Chen, 2013; Chen & Ho, 2012). The STEM pipeline 
continues to be leaky as STEM graduates decide on post-baccalaureate pursuits with only a little 
over half (56%) obtaining employment in a STEM occupation after graduation (Carnevale, Smith, & 
Melton, 2011 ).' 

To address this issue, the United States must seek ways to maintain the STEM pipeline with 
students who are likely to succeed in a STEM major and persist in a STEM field. To this end, we 
developed STEM readiness benchmarks to help identify students who have a high probability of 
succeeding in the first-year mathematics and science courses typically taken by STEM majors. 


1 Results are based on findings from the 1988 National Education Longitudinal Survey (Nels:88; National Center for Education 
Statistics, 1988). Analyses should be replicated with more recent cohorts to determine if these findings represent current trends. 
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The STEM readiness benchmarks will also provide a clear signal to students regarding the rigorous 
demands of the majority of STEM-related coursework. Research shows that many science majors 
have unrealistic expectations about how they will perform in science courses, contributing to an 
overly optimistic view of earning a science degree (Stinebrickner & Stinebrickner, 2014). Many 
students aspire to enter a STEM field, but they are not academically prepared to do so (Noeth, 

Cruce, & Harmston, 2003). For example, of students who completed a core curriculum and intended 
to major in mathematics, less than a third had at least a 50% probability of earning a grade of B or 
higher in a college Calculus class (ACT, 2002). The results for intended science majors revealed that 
the percentage ready for a college-level Chemistry class was higher but was less than half (47%). 

Providing students with an indicator of their STEM readiness has at least two practical benefits. 

First, if provided early enough, students who are interested in STEM but not on track to be ready for 
STEM coursework can take preemptive action to increase the skills and knowledge necessary to 
be STEM-ready. Second, for students making the transition to college, this information may pique 
interest among those who are academically ready for STEM but had not previously considered it as a 
possible major, potentially bolstering the STEM pipeline. By helping students select a major for which 
they are adequately prepared, much of the frustration associated with selecting and enrolling in 
coursework that is a poor academic fit, along with the potential downstream negative consequences 
(e.g., dropping out, switching majors), may be avoided (e.g., Chen & Fio, 2012; Shaw & Barbuti, 2010). 
The goal of this research aligns with the larger objective of the current standards-based educational 
reform movement in the United States, specifically that of the Common Core State Standards 
(CCSS): to clearly articulate the knowledge and skills students need to be ready for college and the 
workforce. 

College and Career Readiness 

Students who graduate from high school ill-prepared for the demands of college-level work have 
been a concern of the United States for decades. Over thirty years ago, the National Commission on 
Excellence in Education released the report A Nation at Risk, which delineated issues and concerns 
with the American education system, namely the diluting of standards, which has had the detrimental 
effect of leaving many students ill-prepared for college-level work and ill-equipped to contribute to 
the economic health of the country (Gardner, Larsen, Baker, & Campbell, 1983). The Commission 
offered several recommendations, one of which was the need to increase the rigor of high school 
graduation requirements. At a minimum, they indicated that all students should be required to 
complete four years of English; three years each of mathematics, science, and social studies; and 
half a year of computer science. 

The report served as a call to action; however, the United States seems to be in a similar 
predicament today with many students earning a high school diploma but lacking the skills 
and knowledge necessary for college-level coursework. The percentage of students requiring 
remediation upon entering college illustrates this point. In the 2007-2008 school year, roughly 20% 
of all first-year undergraduate students took remedial courses in their first year; the rate was even 
higher (24%) among students attending two-year public institutions (U.S. Department of Education, 
2013). Corroborating the results of remedial course taking, a survey of college instructors asked 
to rate the quality of preparation of today's high school graduates indicated that only a quarter of 
college instructors rated the students they teach as “Well” or “Very Well" prepared for college-level 
work (ACT, 2013a). 
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The research clearly indicates a disconnect between the academic skills required for college and 
work and students' actual levels of preparation. In response, various college and career readiness 
benchmarks beyond simply earning a high school diploma have been developed to evaluate and 
track progress toward increasing college and career readiness. One of the earliest examples 
can be traced back to the National Center for Education Statistics (NCES; Berkner & Chavez, 

1997). A college qualification index was developed that included five levels of readiness based 
on the distribution of high school grade point average (HSGPA), class rank, National Education 
Longitudinal Survey (NELS) scores, ACT/SAT scores, and academic coursework among four-year 
college students. The five levels— very highly qualified, highly qualified, somewhat qualified, minimally 
qualified, and marginally or not qualified— corresponded to the performance of four-year college 
students scoring in the top 10%, 25%, 50%, 75%, and the bottom 25% of each of these measures, 
respectively. For example, the cut scores or benchmarks for the very highly qualified category were 
3.7 for HSGPA, 96th percentile for class rank, 97th percentile on NELS, and an ACT score of 28 
or SAT score of 1250. Students only had to achieve one of the criteria (e.g., HSGPA > 3.7) to be 
classified into that level (e.g., very highly qualified). Based on this classification system, 64.5% of 
the 1992 high school graduates were at least minimally qualified, but only 21 .4% were deemed very 
highly qualified. 

The Manhattan Institute for Policy Research also examined college readiness rates using a different 
set of criteria and model for classification (Greene & Winters, 2005). Students were considered 
ready for college if they had earned a high school diploma; completed a minimum set of course 
requirements— four years of English, three years of math, and two years each of natural science, 
social science, and foreign language; and could read at a basic level or above on the National 
Assessment of Educational Progress (NAEP) reading assessment. Unlike the NCES model 
where classification was based on meeting at least one cut score, college readiness was based 
on a conjunctive model where students had to meet all three criteria to be considered college 
ready. Given this distinction, it is not surprising that they concluded that a much smaller percent of 
students were ready for college, just 34%. Both of the definitions of college readiness described 
above set benchmarks irrespective of their relationship with actual college performance. Therefore, 
whether students who were deemed “college-ready” were actually successful once in college and, 
alternatively, whether students deemed not ready were actually unsuccessful remained unanswered. 
To ensure that college readiness indicators were indicative of future college success, others have 
approached the issue by developing college-readiness benchmarks based on empirically derived 
predictive relationships. 

In 2004, a report by ACT, Crisis at the Core: Preparing All Students for College and Work, again 
illustrated that the majority of high school graduates were not ready for college-level work; as a 
result of this analysis, ACT developed empirically derived benchmarks to identify students who had 
a high likelihood of being successful once in college based on their performance on the ACT (ACT, 
2004; Allen, 2013; Allen & Sconing, 2005). Rooting itself in a content validation framework, ACT 
derived subject-specific benchmarks by estimating the ACT subject score associated with a 50% 
probability of earning a grade of B or higher in atypical credit-bearing first-year course completed 
by students in the respective subject. For example, the ACT College Readiness Benchmark for 
Mathematics of 22 was derived based on the relationship between ACT Mathematics scores and 
course grades in College Algebra; for the ACT English, Science, and Reading tests, the courses 
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examined were English Composition, Biology, and social science courses, respectively. 2 Additional 
research on the relationship between the ACT College Readiness Benchmarks and college success 
support the value of these benchmarks: students who meet the ACT benchmarks have higher first- 
year grade point averages (FYGPAs) and are more likely to persist to the second year and, ultimately, 
to graduate from college (e.g., ACT, 2010; Radunzel & Noble, 2012). Among the 2013 ACT-tested 
cohort, roughly one in four (26%) met all four of the ACT College Readiness Benchmarks (ACT, 
2013b). 

In 2007, the College Board released benchmarks specific to the SAT (Kobrin, 2007). Like ACT, 
the College Board calculated its college readiness benchmark based on the relationship between 
SAT scores and performance in the first year of college, though there were slight differences. 

For example, the College Board conceptualized success as earning a FYGPA of a B- or higher 
whereas ACT examined specific content-related course grades. The College Board also employed a 
probability level of 65%; ACT’s was 50%. Using SAT composite score (SAT Math + Critical Reading 
+ Writing), 3 the College Board derived a college readiness benchmark of 1550 (Wyatt, Kobrin, Wiley, 
Camara, & Proestler, 2011 ). Similar to the ACT results, data support the use of the College Board's 
college readiness benchmark in that students who meet the benchmark are more successful in 
college, not only in terms of earning higher FYGPAs but also because they are more likely to enroll 
in college, return for their second year, and graduate in both four and six years (College Board, 2012; 
Kobrin, 2007; Mattern, Shaw, & Marini, 2013; Wyatt et at, 2011). 

The advent of the CCSS and the resulting formation of two multistate consortia, the Partnership 
for Assessment of Readiness for College and Careers (PARCC) and Smarter Balanced, have 
reinvigorated the conversation around college and career readiness (CCSS, 2010). Specifically, the 
two consortia have developed new assessments that will be aligned to the CCSS. The ultimate goal 
is to use the new assessments to evaluate college and career readiness of high school students as 
well as provide diagnostic information at earlier grades regarding whether students are on track for 
college and career readiness; however, the methodology that the two consortia will use to develop 
college and career readiness benchmarks for these new assessments has yet to be determined 
(Camara, 2013; Smarter Balanced Assessment Consortia, 2012). In 2014, ACT launched ACT 
Aspire®, which assesses students' mastery of math, English language arts (ELA), and science in 
grades 3 through 10, allowing for early monitoring of students’ academic strengths and weakness. 
The ACT College and Career Readiness Benchmarks have been backmapped to ACT Aspire, 4 
allowing for the articulation of what students need to know and be able and willing to do at key 
transition points along the K-Career continuum. Such information helps students and teachers know 
if a student is on track for college and career readiness. 


2 The ACT College Readiness Benchmarks were updated in 2013 based on more current data. The new analyses revealed no change 
in the English and Mathematics Benchmarks of 18 and 22, respectively. The Reading Benchmark increased from 21 to 22, and the 
Science Benchmark decreased from 24 to 23 (Allen, 2013). 

3 In 2007, the College Board released benchmarks based on the previous version of the SAT, which did not include the writing section; 
the benchmark was derived based on the composite score of SAT Math + SAT Verbal (Kobrin, 2007). 

4 Benchmarks had previously been used for ACT Explore® (Grades 8 and 9) and ACT Plan® (grade 10). 
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College Readiness is Not STEM Readiness 

The ACT College Readiness Benchmarks provide useful information regarding the extent to which 
students are on track for college and career readiness in a general sense, but they do not address 
(nor were they developed to address) a student’s readiness for a specific college major or career 
field. Research clearly shows that students take different courses in college, and this is due in part 
to varying course requirements across different majors (Berry & Sackett, 2009; Goldman, Schmidt, 
Hewitt, & Fisher, 1974; Pennock-Roman, 1994; Ramist, Lewis, & McCamley-Jenkins, 1994; Westrick, 
2015a). This is particularly true for STEM majors, which are known for having rigorous course 
demands and more stringent grading standards (e.g., Goldman et al., 1974; Westrick, 2015a). In 
fact, the introductory mathematics and science courses for STEM majors are often referred to as 
“gateway" courses, where students who are not academically suited for STEM majors are less likely 
to succeed. Research findings clearly indicate that STEM fields require higher levels of mathematics 
and science knowledge and skills to be successful (Westrick, 2015b); therefore, benchmarks derived 
on the typical courses students take (e.g., College Algebra) may not serve as an adequate indicator 
of an individual student's readiness for STEM coursework. 

Providing a more precise signal to prospective STEM majors may help with the leaky STEM pipeline 
in college. Specifically, the majority of STEM attrition occurs in the first year or two, and readiness for 
college-level STEM coursework is a key factor related to persisting in STEM and ultimately earning 
a STEM degree (Chang, Cerna, Han, & Saenz, 2008; Chen, 2013; Seymour & Hewitt, 1997; Strenta, 
Elliot, Adair, Matier, & Scott, 1994). For example, Chen (2013) found that students in their first 
year who attempted and earned more STEM credits, earned higher grades in their STEM courses, 
withdrew/failed fewer STEM courses, and took Calculus were more likely to persist in STEM. Taking 
Calculus in the first year largely differentiated STEM graduates from non-STEM graduates. Of 
the students who earned a STEM degree, 63% took Calculus or advanced math in the first year 
compared to 28% of students who dropped out of college and 36% of students who switched to 
another major. Among biological, biomedical, and behavior science majors, Chang et al. (2008) found 
that an increase in FYGPA by half a letter grade was associated with roughly a 20% increase in the 
likelihood of persisting in one's major. Likewise, Seymour and Hewitt (1997) found that first-year 
performance was the main determinant of switching out of a STEM major. 

Research has clearly demonstrated that high school graduates who are more academically prepared, 
as measured by high school grades and test scores, are in general more successful in college (ACT, 
2010, 2012a, 2013c, 2013d; Radunzel & Noble, 2012); this is especially true for STEM majors (e.g., 
Chen 2009, 2013; Kokkelenberg & Sinha, 2010; Le, Robbins, & Westrick, 2014; Mendez, Buskirk, 
Lohr, & Haag, 2008; Shaw & Baruti, 2010). In general, research shows that students with higher ACT 
scores are more likely to return for the second year, earn higher grades, and graduate in a timely 
manner (ACT, 2010, 2012b, 2013c, 2013d; Radunzel & Noble, 2012). Research focused on STEM 
majors has reached similar conclusions. Specifically, students with higher grades and test scores 
are more likely to express interest in STEM (ACT, 2012a), enroll in a STEM major (e.g., Goldman 
et al., 1974; Nicholls, Wolfe, Besterfield-Sacre, Shuman, & Larpkiattaworn, 2007; Le et al., 2014; 
Pennock-Roman, 1994; Strenta et al., 1994; Westrick, 2015b), earn higher grades in STEM courses 
(e.g., Chen, 2009, 2013; Chen & Ho, 2012; Mendez et al., 2008), persist in a STEM major (e.g., Chen, 
2009, 2013; Chen & Ho, 2012; Le et al., 2014; Shaw & Baruti, 2010), and graduate with a STEM 
degree (e.g., Chen, 2009; Chen & Ho, 2012; Kokkelenberg & Sinha, 2010). For example, Westrick 
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(2015b) found that among STEM-Quantitative majors who persisted through four years, the average 
ACT Mathematics score for students earning a G PA of 3.0 or higher by semester was 28 across 
four years of college. In contrast, for students who earned semester GPAs of less than 3.0, the 
mean ACT Mathematics score was 24 in the first semester and rose to 26 in the eighth semester 
(due in part to attrition of lower-scoring students). Of note, both the students who averaged a B or 
higher and those who averaged less than a B had substantially higher ACT Mathematics scores, 
on average, than the ACT College Readiness Benchmark in mathematics of 22, suggesting that a 
higher benchmark is needed for STEM majors. 

In sum, the research clearly indicates that STEM majors who are more academically prepared upon 
entry into college are more likely to succeed and persist in a STEM field. What remains unknown is 
the level of academic preparedness students need to have a reasonable chance of succeeding in 
STEM-related courses; that is the focus of the current study. 

Current Study 

The purpose of the current study was to develop indicators or benchmarks of STEM readiness in 
mathematics and science with the goal of informing students about the level of knowledge and 
skills needed to succeed in a STEM major. To accomplish this, three studies were conducted. Given 
that the majority of STEM attrition occurs in the first year or two of college and that it is largely a 
function of performance in the first-year mathematics and sciences courses, the goal of Study 1 
was to determine the typical first mathematics and science courses that STEM majors take or are 
required to take in their first year of college. Building on Study 1 , Study 2 empirically derived STEM 
readiness benchmarks in mathematics and science based on the relationship between ACT scores 
and performance in those STEM-identified first-year mathematics and science courses. Finally, 
Study 3 was conducted to examine the validity of the STEM readiness benchmarks in mathematics 
and science for predicting future success of STEM majors in terms of earning a cumulative GPA of 
3.0 or higher, persisting in a STEM major, and graduating with a STEM-related bachelor's degree. 

Study 1: Course Taking Patterns of STEM Majors 

Sample 

We collected data on first-year courses at 27 public four-year postsecondary institutions from 
three states. 5 Of the 27 institutions, five have highly selective or selective admissions policies. The 
remaining 22 institutions have traditional, open, or liberal admissions policies. The median average 
ACT Composite score across institutions was 21 .4 (1 st Ouartile [01 ] = 20.3; 3rd Ouartile [03] = 
22.5). In terms of student-level data, course transcript data were available for 176,149 students 
who had enrolled in college as first-time entering students from the 2005 through 2009 freshman 
cohorts. 

Course Content Coding. Partnering institutions provided ACT with course grade data, course title, 
and course content codes. 6 For courses that were not assigned a content code by the participating 
institution, ACT research staff assigned a code based on the course title and description from the 


5 The majority of ACT-tested students with intentions to major in a STEM field initially enrolled in a four-year postsecondary institution 
after high school (74%). Therefore, the decision was made to focus on STEM majors at four-year postsecondary institutions. 

6 Course content codes were developed and used to identify similar content-related courses across institutions, e.g., Calculus, College 
Algebra, etc. 
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institution’s course catalog. We then identified for each student the first mathematics and science 
courses taken in the first year. Based on this coding, 100,954 students were identified as taking 
a mathematics course in their first year. Of these students, 3% took more than one mathematics 
course in the same semester. Essentially, these students had multiple “first” mathematics courses in 
college; as such, they were all included in the analyses. For science, 83,403 students were identified 
as taking a science course. Students were much more likely to take more than one science course in 
the same semester— 30%. As was the case with the mathematics results, all of the students’ “first” 
science courses were included in the analyses. 

Coding STEM Majors. Institutions also provided students’ declared majors overtime by reporting a 
six-digit Classification of Instruction Program (CIP) code for each term enrolled. The first-year CIP 
codes were used to identify students who declared a STEM major in their first year. Many definitions 
of STEM exist; the current study used ACT’s definition of STEM (2014), which categorized STEM 
majors into four clusters: Science, Computer Science and Mathematics, Medical and Health, and 
Engineering and Technology. 7 Of the 100,954 students who took a mathematics course in their 
first year, 22% were classified as a STEM major in both the fall and spring terms of Year 1 , which 
translates to 22,113 students. For science, 23,687 of the 83,403 students who took a science 
course in their first year were declared STEM majors in both the fall and spring terms of Year 1 , 
which is 28%. Using this information, the most prevalent first mathematics and science course taken 
in the first year among STEM majors was computed. Characteristics of the students included in the 
mathematics and science sample are summarized in Tables 1 and 2. 


7 Given the lack of consistency among the various STEM definitions being currently employed, ACT conducted a comprehensive review 
of the literature and provided a refined definition of STEM. One characteristic of ACT’s definition is that it excludes social/behavioral 
sciences such as psychology and sociology (Green, 2007). To learn more about which majors and occupations are included in ACT's 
definition of STEM, see the original report (2014b). 
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Table 1. Summary of Student Characteristics of Mathematics Sample for Study 1 

STEM Computer 


Characteristics 

All 

students® 

Overall 

STEM 

STEM 

Science 

Science and 
Mathematics 

STEM Medical 
and Health 

STEM Engineering 
and Technology 

Non-STEM 

students 

N (students) 

100,954 

22,113 

8,413 

2,821 

3,780 

7,099 

64,657 

Gender 

Female 

52% 

45% 

55% 

27% 

82% 

19% 

55% 

Male 

45% 

53% 

42% 

71% 

16% 

78% 

42% 

Missing 

2% 

00 

o" 

2% 

CM 

2% 

3% 

2% 

Race/Ethnicity 

African American 

14% 

12% 

10% 

18% 

22% 

CO 

14% 

Asian American 

3% 

4% 

CJl 

CO 

2% 

5% 

2% 

Hispanic 

9% 

12% 

14% 

12% 

LO 

13% 

9% 

Other 

CD 

5% 

CJl 

4% 

LO 

4% 

6% 

White 

62% 

60% 

59% 

56% 

62% 

63% 

62% 

Missing 

7% 

7% 

7% 

7% 

5% 

9% 

6% 

ACT Composite score 

Mean 

21.8 

23.3 

23.4 

22.8 

20.3 

25.0 

21.5 

Standard deviation 

4.7 

4.9 

4.7 

5.2 

4.2 

4.8 

4.5 

HSGPA 

Mean 

3.40 

3.55 

3.60 

3.41 

3.36 

3.64 

3.38 

Standard deviation 

0.53 

0.48 

0.43 

0.57 

0.53 

0.41 

0.52 

Missing 

17% 

15% 

14% 

16% 

13% 

16% 

17% 


Note. Percentages may not sum to 100% due to rounding. 

a The number of students in this column is not the sum of the overall STEM column and the non-STEM students column. Students who were STEM 
majors for only one term in their first year were in the total group but not included in the subanalyses, given that they did not cleanly fall into either 
category. 
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Table 2. Summary of Student Characteristics of Science Sample for Study 1 


STEM Computer 


Characteristics 

All 

students® 

Overall 

STEM 

STEM 

Science 

Science and 
Mathematics 

STEM Medical 
and Health 

STEM Engineering 
and Technology 

Non-STEM 

students 

N (students) 

83,403 

23,687 

10,140 

1,708 

4,370 

7,469 

49,053 

Gender 

Female 

54% 

47% 

56% 

29% 

82% 

19% 

57% 

Male 

44% 

50% 

41% 

69% 

16% 

78% 

40% 

Missing 

3% 

3% 

3% 

2% 

2% 

3% 

3% 

Race/Ethnicity 

African American 

11% 

9% 

9% 

18% 

14% 

5% 

12% 

Asian American 

3% 

5% 

6% 

3% 

2% 

5% 

CO 

Hispanic 

8% 

11% 

13% 

12% 

LO 

11% 

8% 

Other 

6% 

5% 

5% 

4% 

5% 

4% 

6% 

White 

65% 

63% 

60% 

55% 

70% 

65% 

65% 

Missing 

7% 

7% 

7% 

00 

4% 

9% 

7% 

ACT Composite score 

Mean 

22.8 

24.0 

23.9 

23.3 

21.5 

25.8 

22.4 

Standard deviation 

4.6 

4.7 

4.5 

5.3 

4.0 

4.6 

4.5 

HSGPA 

Mean 

3.51 

3.62 

3.64 

3.47 

3.48 

3.70 

3.48 

Standard deviation 

0.48 

0.43 

0.41 

0.54 

0.47 

0.36 

0.49 

Missing 

16% 

14% 

14% 

17% 

12% 

16% 

16% 


Note. Percentages may not sum to 100% due to rounding. 

a The number of students in this column is not the sum of the overall STEM column and the non-STEM students column. Students who were STEM 
majors for only one term in their first year were in the total group but not included in the subanalyses given that they did not cleanly fall into either 
category. 
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Results 

The percentage of students who took a mathematics and science course by content area was 
computed for the total sample and by STEM major category. 8 Specifically, the most typical first-year 
mathematics and science course was examined for all STEM majors and for each of the four STEM 
clusters. This was compared to all students and non-STEM majors. The results for mathematics and 
science are summarized in Tables 3 and 4, respectively. 

Mathematics Results. Across all students in the sample, regardless of their major, College Algebra 
was the most typical course (28%). For non-STEM majors, the percentage was slightly higher 
(31%). Moreover, an additional 21% of all students first enrolled in courses below College Algebra; 
courses often considered developmental, non-credit-bearing courses. These results coincide with 
the ACT College Readiness Benchmark research that derived the ACT Mathematics Benchmark on 
course grades in College Algebra (Allen, 2013; Allen & Sconing, 2005). However, when focusing 
exclusively on STEM majors, a different pattern emerges. Namely, over a quarter of STEM majors 
took Calculus I as their first mathematics course (28%); another 10% took Calculus II as their 
first course. Only 11% of non-STEM majors took Calculus I as their first mathematics courses; for 
another 2%, it was Calculus II. Clearly, STEM majors are more frequently enrolling in higher-level 
mathematics courses compared to non-STEM majors. 


Weighted percentages are reported. The sample was weighted to be similar to an ACT-tested, four-year college enrollee population 
with respect to institution admission selectivity (less versus more selective). The selectivity of admission policies was self-reported 
by the institutions using five levels that classified their level according to the typical ACT Composite score and high school ranks 
of their accepted freshmen. The population included a larger proportion of more selective institutions (39% selective versus 61% 
less selective) as compared to the sample (19% selective versus 81% less selective). See Tables 5 and 6 for more details about the 
population. 
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Table 3. Distribution (%) of Students’ First College Mathematics Course by STEM Category 


STEM Computer 


Mathematics content area 

All 

students 

Overall 

STEM 

STEM 

Science 

Science and 
Mathematics 

STEM Medical 
and Health 

STEM Engineering 
and Technology 

Non-STEM 

students 

Business Mathematics 

5 

3 

4 

5 

3 

1 

6 

Arithmetic 

1 

0.3 

0.3 

0.4 

1 

0 

1 

Elementary Algebra 

8 

5 

4 

7 

15 

1 

9 

Intermediate Algebra 

12 

6 

7 

7 

13 

2 

13 

College Algebra 

28 

18 

22 

15 

48 

5 

31 

Geometry 

0 

0 

0 

0 

0 

0 

0 

Analytical Geometry 

1 

2 

3 

2 

1 

0.1 

1 

Trigonometry 

4 

7 

9 

7 

2 

7 

3 

Precalculus 

6 

5 

6 

11 

0.5 

4 

6 

Calculus 1 

15 

28 

25 

28 

3 

38 

11 

Statistics 

2 

2 

3 

1 

3 

1 

2 

Logic 

3 

1 

1 

5 

0.4 

0.3 

4 

Other 

10 

10 

8 

8 

8 

13 

10 

Contemporary Mathematics 

2 

3 

6 

1 

3 

0.3 

2 

Survey Calculus 

3 

7 

3 

5 

1 

13 

2 

Calculus II 

4 

10 

6 

8 

0.1 

18 

2 

Multiple courses 

3 

4 

5 

9 

2 

3 

2 


Note. Data are based on 27 public four-year, postsecondary institutions from three states (2005 through 2009 freshman cohorts). There were 100,954 total 
students, 22,113 first-year STEM majors, and 64,657 first-year non-STEM majors. The breakdown by STEM cluster was: 8,413 for science; 2,821 for computer 
science and mathematics; 3,780 for medical health; and 7,099 for engineering. Examples of courses included in the “Other” category include Elementary 
Functions, Introduction to Mathematics, and Linear Equation Inequalities. Percentages will sum to above 100% due to the fact that some students took multiple 
mathematics courses in the same term. 
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Table 4. Distribution (%) of Students’ First College Science Course by STEM Category 

STEM Computer 


Science content area 

All 

students 

Overall 

STEM 

STEM 

Science 

Science and 
Mathematics 

STEM Medical 
and Health 

STEM Engineering 
and Technology 

Non-STEM 

students 

General Science 

1 

0.3 

0.1 

2 

1 

0.1 

1 

Biology 

34 

31 

47 

26 

48 

8 

35 

Chemistry 

35 

54 

60 

25 

40 

57 

24 

Physics without Calculus 

2 

2 

1 

3 

0.4 

3 

1 

Physics with Calculus 

2 

4 

2 

4 

0.1 

8 

1 

Botany 

1 

1 

1 

1 

0 

0.1 

1 

Ecology 

1 

1 

1 

0.3 

0 

0.2 

0.5 

Engineer 

10 

23 

2 

5 

1 

57 

4 

Zoology 

6 

5 

10 

3 

2 

0.5 

6 

Anatomy 

2 

2 

1 

1 

11 

0 

2 

Health Science 

21 

13 

14 

18 

38 

2 

25 

Astronomy 

2 

1 

0.5 

4 

0.1 

0.4 

3 

Geology 

7 

3 

2 

10 

0.3 

5 

10 

Meteorology 

1 

1 

3 

1 

0 

0.1 

1 

Other 

9 

15 

7 

19 

1 

27 

7 

Biology II 

0.3 

1 

1 

0.4 

0.4 

0.3 

0.2 

Chemistry II 

1 

1 

1 

0.3 

0.3 

1 

0.3 

Multiple courses 

30 

49 

47 

19 

37 

60 

20 


Note. Data are based on 27 public four-year institutions from three states (2005 through 2009 freshman cohorts). There were 83,403 total students, 
23,687 first-year STEM majors, and 49,053 first-year non-STEM majors. The breakdown by STEM cluster was: 10,140 for science; 1 ,708 for computer 
science and mathematics; 4,370 for medical health; and 7,469 for engineering. Examples of courses included in the “Other" category include Physical 
Science, Earth Science, and Space Science. Percentages will sum to above 100% due to the fact that some students took multiple science courses in the 
same term. 


That being said, a sizable percentage of STEM majors first enrolled in lower mathematics courses, 
including 5% for Elementary Algebra. Such findings highlight the misalignment between career 
aspirations and academic preparation for some students and may explain some of the STEM attrition 
occurring at the postsecondary level. This is supported empirically where research has consistently 
found that students in developmental courses are less successful in college than their non- 
developmental peers (Noble & Sawyer, 2013). 

The mathematic results were also examined by STEM cluster, and the results largely mirror the 
overall STEM results with Calculus being the most typical first mathematics course taken for the 
Science, Computer Science and Mathematics, and Engineering and Technology clusters. 9 The only 
cluster that diverged from these findings was the Medical and Health cluster where College Algebra 
was the most typical first mathematics course taken. Previous research has shown that students 
interested in these fields tend to be less academically prepared (ACT, 2014b). It should also be 
pointed out this cluster comprises a small percentage of STEM majors at four-year institutions 


9 For these three STEM clusters combined, 32% of STEM students took Calculus I as their first mathematics course and an additional 
12% took Calculus II. 
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as well as inclusive of majors not typically thought of as traditional STEM majors (e.g., Medical 
Laboratory Technology; for a complete list of the majors included in this cluster, see ACT, 2014). In 
the current sample, 70% of the students in the Medical and Health Cluster were majoring in nursing. 

Science Results. Examination of students’ first science course indicated that Biology and 
Chemistry were the most prevalent courses taken across all students (see Table 4). Focusing on 
non-STEM majors, Biology was more typical (35%) than Chemistry (24%). For STEM students, 
the most typical first science course taken was Chemistry (54%), followed by Biology (31%), and 
Engineering (23%). Moreover, nearly half (49%) had more than one “first” science course compared 
to 20% of non-STEM students. Disaggregating the results by STEM cluster, Chemistry was the most 
typical course for the STEM Science cluster, whereas Engineering and Chemistry were equally likely 
for the STEM Engineering and Technology cluster (57%). The STEM Engineering and Technology 
cluster had the largest percentage of students with multiple first science courses (60%). For the 
STEM Computer Science and Mathematics cluster, Biology and Chemistry were the most typical 
courses with 26% and 25%, respectively. Finally, for the STEM Medical and Health cluster, Biology 
was the most typical first science course. 

Course Requirement Results. To supplement these findings, first-year mathematics and science 
course requirements of the most popular STEM majors were coded from course catalogs of college 
and universities that admitted the largest percentages of ACT-tested students who intended on 
majoring in STEM. 10 Specifically, the top 10 institutions that enrolled the most ACT-tested students 
who were planning to major in STEM for each of the four STEM clusters were identified. For the top 
three to four majors within each cluster, * 11 the first-year course requirements in mathematics and 
science were coded from the course catalog. Based on a sample of 90 STEM major-by-institution 
comparisons, 79% indicated Calculus as a first year requirement. The first-year science requirements 
varied more as a function of the field of study; however, 90% indicated Chemistry, Biology, Physics, 
and/or Engineering as satisfying first-year requirements. These findings coincide with the empirical 
course-taking results summarized above with the exception of Physics, which was more frequently 
identified in these supplemental analyses. The findings from both analyses informed the selection of 
college courses used to derive STEM readiness benchmarks in mathematics and science. 

Study 2: Development of STEM Readiness Benchmarks 

Sample 

Course grade data used in the current study were provided by four-year postsecondary institutions 
that have participated in research services offered by ACT, including state partnerships, the Course 
Placement Service, and Prediction Service. Data included in the analyses were limited to students 
from the 2005 through 2009 freshman cohorts. The number of institutions and students per 
analysis varied based on the data available by course content area; this information along with 
additional information on key institutional and student characteristics of the sample is summarized in 
Tables 5 and 6. Similar to Study 1, students’ first mathematics and science courses taken during the 
first year were used in the analyses. 12 


10 These colleges and universities were determined using college enrollment records from the National Student Clearinghouse for all 
2013 ACT-tested high school graduates who indicated that they planned to major in a STEM field (A/ = 530,428 students). 

11 Majors with the highest numbers of students interested in them by cluster were identified from ACT's Condition of STEM report 
(2014). 

12 This criterion is also used in ACT Course Placement Research services to avoid having intervening coursework influence test score- 
course outcome relationships. 
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Table 5. Summary of Institutional Characteristics of Study 2 


Characteristic 

Calculus 

Combined Science 

Population 

# of four-year institutions 

37 

70 

1,251 

Admissions policy 

Highly selective, selective 

43% 

26% 

39% 

Traditional, open, liberal 

57% 

74% 

61% 

Sector 

Private 

11% 

29% 

60% 

Public 

89% 

71% 

40% 

Region 

East 

14% 

16% 

47% 

Midwest 

22% 

24% 

26% 

Southwest 

51% 

46% 

10% 

West 

14% 

14% 

16% 

Average ACT Composite score 

Median 

22.4 

21.8 

22.4 

1 st quartile to 3rd quartile 

21.5 to 25.0 

20.9 to 23.4 

20.5 to 24.1 


Note. Combined Science courses include Biology (58 institutions), Chemistry (44 institutions), Physics with Calculus 
(7 institutions), and Engineering (6 institutions). Thirty-nine of the institutions had grades available for only one science 
course; 31 institutions had grades available for more than one science course (three institutions had all four; seven had 
three courses; 21 had two courses). Population includes four-year postsecondary institutions where 2012 ACT-tested 
graduates initially enrolled in fall 2012 (determined using enrollment records from the National Student Clearinghouse). 


Analyses 

The same methodology employed to derive the ACT College Readiness Benchmarks was used in 
the current study (Allen & Sconing, 2005; Allen, 2013). The distinction from the previous research 
is that the benchmarks were derived based on performance in the STEM-identified mathematics 
and science courses from Study 1 instead of the most typical course among all students. In 
mathematics, performance in Calculus was examined. In science, the most common course taken 
varied by the field of study; therefore, a combination of science courses was examined: Chemistry, 
Biology, Physics, and Engineering. Course success was defined as earning a grade of a B or higher. 
To account for students being nested within institutions, hierarchical logistic random intercepts and 
slopes regression models 13 were run to estimate the following relationships between scores on the 
ACT and course success: 

■ ACT Mathematics scores and success in Calculus. 

■ ACT Science scores and success in Chemistry, Biology, Physics, and/or Engineering. 


13 Student weighting was employed to ensure that the sample was representative of a larger population of ACT-tested high school 
graduates in terms of race/ethnicity, ACT Composite score, and HSGPA. This is the population to which the ACT College Readiness 
Benchmarks are reported on and thus the envisioned group to whom the STEM-specific benchmarks would be conveyed to (if not 
earlier). 
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Table 6. Summary of Student Characteristics of Study 2 


Characteristics 

Calculus 

Combined Science 

Population 

N (students) 

22,246 

69,328 

1,168,752 

Gender 

Female 

43% 

53% 

53% 

Male 

54% 

46% 

47% 

Missing 

3% 

2% 

0.3% 

Race/Ethnicity 

African American 

3% 

8% 

15% 

Asian American 

5% 

4% 

00 

Hispanic 

CD 

6% 

10% 

Other 

4% 

3% 

4% 

White 

74% 

73% 

63% 

Missing 

CO 

6% 

LO 

ACT Composite score 

Mean 

25.5 

23.9 

20.7 

Standard deviation 

3.8 

4.3 

5.1 

HSGPA 

Mean 

3.66 

3.60 

3.17 

Standard deviation 

0.37 

0.43 

0.65 


Note. Combined Science courses include Biology (29,784 students), Chemistry (36,382 students), Physics 
w/Calculus (629 students), or Engineering (2,533 students). Population includes ACT-tested high school 
graduates of 2012 from states where at least 50% of students took the ACT. 


The NLMIXED procedure within SAS 9.2 statistical software was used to fit the models and obtain 
the parameter estimates for each college. 

For Calculus, the estimated intercepts and slopes were used to calculate institution-specific ACT 
Mathematics scores associated with a 50% probability of success in Calculus. Data permitting, the 
estimated intercept and slope derived for each science course (i.e., Chemistry, Biology, Physics, and 
Engineering) within each institution was used to compute the institution-specific ACT Science score 
associated with a 50% probability of success in each science content area. For each institution, 
the median ACT Science score of the within-institution science results was computed. Finally, after 
apply institutional weighting, 14 the median STEM readiness benchmarks in mathematics and science 
across institutions was computed. 


14 Institutional weighting consisted of applying weights to the institution specific cut scores to make each sample similar to an ACT- 
tested four-year-college-enrollee population with respect to both institutional selectivity and size. Similar results were obtained when 
weighting only took into account institutional selectivity and did not give more weight to larger institutions. 
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Results 

Descriptive results. The distribution of course grades in Calculus and each of the science courses 
was examined. For Calculus, 53% of students earned a grade of a B or higher in the course. Across 
the four science courses examined, roughly half (49%) of the students earned a B or higher, 
though it did vary by content area with more students earning a B or higher grade in Physics and 
Engineering than Chemistry and Biology. Specifically, 69% of students earned a grade of a B or 
higher in Physics and Engineering compared to 50% for Chemistry and 45% for Biology. 

Hierarchical logistic regression results. Calculus results. First-year Calculus grades were 
available on over 20,000 students attending one of 37 four-year institutions. 15 The median ACT 
Mathematics score associated with a 50% probability of earning a B or higher grade in Calculus 
is 27 (Q1 = 25; Q3 = 28; see Table A1 for parameter estimates). This is consistent with previous 
research on the relationship between ACT Mathematics scores and Calculus grades (ACT, 2014a). 
Graphically illustrating the positive relationship between performance on the ACT Mathematics test 
and grades earned in Calculus courses, Figure 1 highlights that higher ACT Mathematics scores 
are associated with a higher likelihood of success at typical postsecondary institutions. For example, 
students who score at the top of the ACT Mathematics score range have an over 80% probability 
of earning a grade of B or higher in Calculus. On the other hand, students with an ACT Mathematics 
score of 22, which is the ACT College Readiness Benchmark, have only a 32% chance of similar 
success in Calculus. The probability of earning a C or higher is also plotted. For students with an 
ACT Mathematics score of 27 or above, students’ chances of earning a C or higher is at least 75% 
at typical four-year postsecondary institutions. 

Combined Science results. Chemistry, Biology, Physics, and Engineering course grade data were 
available for a larger number of students. 16 Namely, the analyses for the combined science courses 
are based on nearly 70,000 students attending one of 70 four-year institutions. The median ACT 
Science score associated with a 50% probability of earning a B or higher grade in Chemistry, 
Biology, Physics, or Engineering is 25 (Q1 = 24; Q3 = 27). Figure 2 graphically displays the 
increasing likelihood of success in one of these science courses as ACT Science scores increase 
(parameter estimates are provided in Table A1). As was the case with the Calculus results, students 
who score at the top end of the ACT Science score scale are very likely to be successful in STEM- 
identified science courses. 

The results from Study 2 strongly suggest that higher ACT Mathematics and Science scores are 
needed for students to have a reasonable chance of being successful in typical first-year subject- 
related courses taken by STEM majors. 


15 All students who took Calculus I as their first mathematics course were included in the analyses. Students taking a Survey of Calculus 
course were not included. The sample was not restricted to STEM majors. Given the concern that Calculus courses taken by STEM 
majors may be more difficult than Calculus courses taken by non-STEM majors, follow-up analyses were conducted to determine 
whether differences would arise if the sample had been restricted to STEM majors. Identical results were found, supporting the 
generalizability of the findings. 

16 Additional data cleaning of the science course data was conducted. Specifically, only science courses that fulfilled course 
requirements for science majors were analyzed; science courses that were offered to non-science majors were excluded. This included 
the removal of 2,427 student records for Chemistry and 9,784 student records for Biology. 
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ACT Mathematics score 


Figure 1 . Probability of Success in Calculus I by ACT Mathematics Score at Typical Four- 
Year Institutions 



ACT Science score 


Figure 2. Probability of Success in Chemistry, Biology, Physics, or Engineering Courses by 
ACT Science Score at Typical Four-Year Institutions 
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Study 3: Validation of STEM Readiness Benchmarks 

Study 3 examined whether STEM majors who met the STEM benchmarks were more successful 
later in college than students who did not. Multiple indicators of later success in college were 
examined: earning a cumulative GPA of 3.0 or higher in Years 1 through 4, persisting in a STEM 
major to the second, third, and fourth year, and graduating with a STEM degree in 4, 5, or 6 years. 

Sample 

Longitudinal college outcomes data used in the current study were provided by four-year 
postsecondary institutions who have participated in research services offered by ACT. Data included 
in the analyses were limited to students who were first-year STEM majors from the 2005 through 
2009 cohorts, who were tracked primarily at their initial institution for at least four years. 17 College 
outcomes data were available from 48 institutions, representing 53,109 STEM majors. Of these 
STEM majors, 32% met the STEM mathematics benchmark and 39% met the STEM science 
benchmark. Scores on the ACT Mathematics and Science tests are highly correlated (7 = 0.77); 
therefore, the percentage meeting both or neither is quite high (80%) with 55% meeting neither 
and 25% meeting both. Of students who met the STEM mathematics benchmark (n = 16,932), the 
majority met the STEM science benchmark (78%); however, only 65% of students who met the 
STEM science benchmark also met the STEM mathematics benchmark. Additional information on 
key institutional and student characteristics of the sample is summarized in Tables 7 and 8. 


17 First-year STEM majors included 49,917 (94%) who declared a STEM major during their first fall term and 3,192 (6%) who were 
undeclared during their first fall term but declared a STEM major during their first spring term. Longitudinal data was available on 
STEM majors entering college in 2005 through 2009. For earlier cohorts, up to eight years of outcome data were available. For the 
most recent cohorts, at least four years of longitudinal data were available. For two state systems, students were tracked across in- 
state institutions. 
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Table 7. Summary of Institutional Characteristics 
of Study 3 


Table 8. Summary of STEM Student Characteristics of 
Study 3 


Characteristic 

Study 3 

Characteristic 

Study 3 

N (four-year institutions) 

48 

N (students) 

53,109 

Admissions policy 


Gender 


Highly selective, selective 

25% 

Female 

48% 

Traditional, open, liberal 

75% 

Male 

49% 

Sector 


Missing 

3% 

Private 

27% 

Race/Ethnicity 


Public 

73% 

African American 

10% 

Region 


Asian American 

4% 

East 

13% 

Hispanic 

9% 

Midwest 

10% 

Other 

5% 

Southwest 

73% 

White 

64% 

West 

4% 

Missing 

CO 

Average ACT Composite score 


ACT Mathematics score 


Median 

21.6 

< 27 

68% 

1 st quartile to 3rd quartile 

20.4 to 23.3 

> 27 (met STEM mathematics benchmark) 

32% 



ACT Science score 


Note. The typical (median) number of first-year STEM majors per 



institution was 624 students (ranged from 70 to 4,541 students). 

< 25 

61% 


> 25 (met STEM science benchmark) 

39% 

Academic performance 

Mean (SD) 

ACT score 


Composite 

23.7 (4.9) 

English 

23.4 (5.7) 

Mathematics 

23.8 (5.3) 

Reading 

23.8 (5.9) 

Science 

23.3 (4.8) 

HSGPA 


Overall 

3.56 (0.46) 

Math 

3.5 (0.6) 

Science 

3.6 (0.5) 
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Analyses 

Due to the nested structure of the data, various hierarchical logistic regression models were used 
to estimate students’ chances of succeeding in a STEM major at typical four-year institutions. To 
examine cumulative course performance, we used hierarchical logistic regression models to estimate 
the probability of earning a 3.0 (equivalent to a grade of a B) or higher cumulative GPA at the end of 
Year 1 through the end of Year 4 as a function of whether the STEM readiness benchmark was met. 
Parameter estimates for both the intercept and benchmark indicator (met/not met) were allowed to 
vary randomly across institutions. Analyses of cumulative grades beyond Year 1 were based on the 
subsample of students who persisted in a STEM major to ensure that a majority of their grades were 
earned in STEM-related courses. For students who were not enrolled during the second term of their 
first year, their GPAs from the first term were carried forward and used in the Year 1 cumulative GPA 
analyses. 

For persistence in STEM, a hierarchical multinomial regression model employing a logit link was 
used to evaluate students' chances of three distinct outcomes: persisting in a STEM major, switching 
to a non-STEM major, and dropping out of the institution. Persisting in a STEM major was the 
reference category. For each time point (Year 2 to Year 4), the three-category STEM persistence 
outcome was based on students' declared majors during the spring semester of the corresponding 
year. 18 Students who were no longer enrolled but who had completed a bachelor’s degree were 
categorized as persisting in STEM or switching to non-STEM according to their degree major. 
Intercepts were allowed to vary randomly across institutions. 19 

Hierarchical discrete-time survival models under the proportional hazards assumption were 
developed to predict STEM-related bachelor’s degree completion from the STEM readiness 
indicators (Singer & Willett, 1993; Reardon, Brennan, & Buka, 2002). This approach simultaneously 
models all time periods while also accounting for censored observations due to the various freshman 
cohorts being tracked for different lengths of time. In these models, the logit of the conditional 
probability of degree completion in a particular term, given that no degree was earned prior to that 
term, was modeled as a linear function of term indicators and the STEM readiness benchmark 
indicator. The discrete-time analyses focused on fall and spring terms. There were very few degrees 
given in the summer terms; summer term degree completion was therefore combined with that for 
the prior spring term. Term indicators for Term 6 (spring/summer term of Year 3) through Term 12 
(spring/summer term of Year 6) were included in the models. Parameter estimates for the term 
indicators and the STEM readiness indicator were allowed to vary randomly across institutions. 

The GLIM MIX procedure for generalized mixed models, available in SAS 9.2, with the Laplace 
estimation method was used to fit the models. In both mathematics and science, the STEM 
readiness indicator was a statistically significant predictor for all outcomes across all time points, 

p < 0.0001 . 


18 Institutions provided six-digit CIP codes over time for each term enrolled. 

19 For persistence in STEM, a random intercept and slope model did not converge for either STEM readiness benchmark for all 
outcomes, most likely because the variance estimate of the STEM readiness indicator was close to 0. For the instances where the 
results did converge, the random intercept model and the random intercept and slope model provided nearly identical results. Results 
for the random intercept model are reported. 
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Results 


Cumulative GPA. Summarized in Table 9, the results indicate that the likelihood of earning a 
cumulative GPA of 3.0 or higher was strongly related to meeting the STEM benchmarks (see 
Table A2 for parameter estimates for the models). For students meeting the STEM mathematics 
benchmark, the chance of earning a cumulative GPA of 3.0 or higher in Year 1 was 71%. 

Their chances rose to a high of 80% in Year 4. In contrast, for students not meeting the STEM 
mathematics benchmark, the chances of earning a cumulative GPA of 3.0 or higher across the four 
years was much lower, ranging from a low of 39% for Year 1 to a high of 57% in Year 4. A similar 
pattern of results emerged by STEM science benchmark attainment. 

Table 9. STEM Students’ Estimated Probability of Earning a Cumulative GPA of 3.0 or higher 
by STEM Benchmark Attainment 


Year 1 Year 2 Year 3 Year 4 



N 

Prob. 

N 

Prob. 

N 

Prob. 

N 

Prob. 

STEM mathematics benchmark (27) 

Not met 

34,291 

.39 

18,095 

.48 

14,598 

.53 

12,991 

.57 

Met 

15,661 

.71 

11,495 

.76 

10,363 

.78 

9,807 

.80 

STEM science benchmark (25) 

Not met 

30,995 

.38 

16,473 

.46 

13,331 

.51 

11,882 

.55 

Met 

18,957 

.64 

13,117 

.73 

11,630 

.75 

10,916 

.77 


Note. Cumulative GPA for Year 2 and beyond are based only on students who remained in a STEM field. N = sample size; 
Prob. = probability. 
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Persistence in STEM. As was the case for cumulative GPA, students who met the STEM readiness 
benchmarks were more likely to persist in a STEM major. As displayed in Figure 3, students' chances 
of persisting in a STEM major to Year 2 was 71 % for those who met the STEM mathematics 
benchmark compared to 52% for those who did not met the STEM benchmark (parameter estimates 
are provided in Table A3). STEM benchmark attainment was also related to students' chances of 
not being enrolled at the institution at the end of Year 2: 29% of those who were not STEM ready in 
mathematics as compared to 17% for those who were STEM ready in math. Students' likelihood of 
switching to a non-STEM major by academic year is also provided. In Year 2, the likelihood was 19% 
for those who were not STEM ready in mathematics compared to 11 % for those who were STEM 
ready in math. By Year 4, the chances of students persisting in a STEM major, switching into a non- 
STEM major, and no longer being enrolled at that institution were 59%, 13%, and 28%, respectively, 
for those who were STEM ready in math and 36%, 21 %, and 43%, respectively, for those who were 
not. A similar pattern emerged when examining persistence rates by STEM science benchmark 
attainment status. Namely, STEM attrition occurred more frequently for students who did not meet 
the STEM readiness benchmark in Science. 



Not Enrolled 

■ Left STEM 

■ Persisted 


Figure 3. Probability of Persisting in a STEM Field Compared to Leaving STEM or Being No 
Longer Enrolled at the Institution by STEM Benchmark Attainment 
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STEM-related bachelor’s degree. Finally, the results indicate that students entering college 
academically ready for STEM coursework were more likely to earn a STEM-related bachelor’s 
degree compared to STEM majors who did not meet the STEM readiness benchmarks (see Figure 
4 and Table A4). By the end of six years, students' chances of earning a STEM-related bachelor’s 
degree was roughly half (49%) for students who met the STEM mathematics benchmark. In fact, 
these students were more than twice as likely to earn a STEM-related bachelor's degree than 
students who did not meet the STEM mathematics benchmark (23%). For the STEM science 
benchmark, students’ chances of earning a STEM-related bachelor's degree was 42% for those 
who met the benchmark and 22% for those who did not. Additional analyses were run to model 
students’ probability of being successful for the various outcomes if they met both STEM readiness 
benchmarks, and the results indicated that students who met both were more likely to be successful 
than students who met only one or neither. 



Figure 4. Probability of Completing a STEM-Related Bachelor's Degree in Four, Five, and Six 
Years by STEM Benchmark Attainment 

In sum, the STEM readiness benchmarks not only provide information about students' likelihood 
of earning a grade of a B or higher in typical first-year mathematics and science courses taken by 
STEM majors, but they are also related to a variety of important and long-term academic outcomes 
such as earning a cumulative GPA of 3.0 or higher through the fourth year of college, persisting in a 
STEM major, and ultimately earning a STEM-related bachelor's degree. 
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Discussion 

The current study builds on the literature on STEM majors and academic success. Namely, to the 
best of the authors' knowledge, this is the first study to examine the first-year course-taking patterns 
of STEM majors using actual course transcript data as well as institution course catalog program 
requirements. The results clearly indicate that STEM majors are more likely than typical students to 
take higher-level mathematics courses beyond College Algebra (namely, Calculus) during their first 
year in college. In science, multiple courses were identified as the typical first-year science course 
taken by STEM majors, with the most prevalent course tending to vary somewhat by STEM cluster. 
These courses included Biology, Chemistry, Engineering, and Physics. Based on students' actual 
course grades, the level of proficiency needed in mathematics and science to ensure a reasonable 
probability of success (50% of a B or higher) in their first-year, STEM-related mathematics and science 
courses was estimated. The results underscore the fact that a higher level of academic preparation 
is needed than that of the typical college student, as indicated by higher cut scores for the STEM 
readiness benchmarks than the general ACT College Readiness Benchmarks in mathematics and 
science. We also evaluated the importance of meeting the STEM readiness benchmarks for success 
in STEM beyond first-year mathematics and science courses. The findings indicate that meeting the 
STEM readiness benchmarks is associated with greater chances of earning a cumulative GPA of 3.0 
or higher over time, persisting in a STEM major, and earning a STEM-related bachelor’s degree. 

Providing students with this type of information early on to help them determine whether they 
are prepared to major in STEM will hopefully result in more students entering the field who are 
academically ready for its rigorous demands, thereby potentially avoiding the negative consequences 
associated with enrolling in a program that is a poor academic fit (e.g., switching major). Specifically, 
the STEM readiness benchmarks can be backmapped to the ACT Aspire score scale, providing the 
opportunity to give students information on their level of readiness for STEM as early as third grade. 
Additionally, this information may help spark interest in STEM among those who are academically 
prepared, potentially bolstering the STEM pipeline. 

One limitation of the current study is its exclusive focus on the cognitive determinants of STEM 
success. Research clearly suggests that performance in school and on the job is jointly influenced 
by both cognitive and non-cognitive factors (e.g., Allen & Robbins, 2010; Barrick & Mount, 1991 ; 

Le et at, 2014; Nye, Su, Rounds, & Drasgow, 2012; Poropat, 2009; Richardson, Abraham, & Bond, 
2012; Robbins, Lauver, Le, Davis, Langley, & Carlstrom, 2004; Van Iddekinge, Putka, & Campbell, 

2011 ). Not specific to STEM majors but to all college students, Allen and Robbins (2010) found that 
motivation (as measured by ACT Engage®) and interest-major congruence was predictive of timely 
degree completion above and beyond academic preparation. Examining the role of motivational 
factors, students’ interests in STEM, and their high school coursework and grades in conjunction 
with test scores to better understand STEM success seems like a fruitful avenue. Though out of the 
current paper’s scope, preliminary analyses were run examining the incremental validity of STEM 
interests over mathematics and science ability in predicting STEM success. Using the sample of 
STEM majors from Study 3, the results indicate that expressed interest in STEM (i.e., student’s 
intentions to major in STEM as a high school student) and measured interests in STEM (based on 
responses to the ACT Interest Inventory [2009], which was initially developed by Holland and based 
on his theory of vocational interests [1997]) did add incrementally to the prediction of all three 
measures of success: cumulative GPA, persistence, and degree completion in STEM— though the 
effect for cumulative GPA was small. In terms of degree completion, a student’s chance of earning 
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a STEM-related bachelor's degree by Year 6 for those who met the STEM mathematics benchmark 
and had both expressed and measured interest in STEM was 50%; it was 43% for those who met 
the benchmark but had neither expressed nor measured interest in STEM. 

That being said, single-dimension measures of readiness have advantages as well. Single-dimension 
measures allow for monitoring over time of distinct dimensions of readiness. One of the primary uses 
of the ACT College Readiness Benchmarks is to provide targets for students in high school (and 
earlier grades) to work toward. The benchmark concept could be extended to multiple-dimension 
measures (e.g., based on a dichotomization of a readiness index that incorporates multiple measures 
such as high school courses taken and grades earned, measures of motivation and engagement, 
interest data, and ACT scores). While multiple-dimension measures are more predictive of college 
outcomes, single-dimension benchmarks are still useful because they are more informative of a 
student's specific areas of need. 

Another limitation of the current study is the representativeness of the samples used to develop and 
validate the STEM readiness benchmarks. Even though the current study employed large, multi- 
institutional samples of students and colleges, colleges in the southwest were overrepresented in 
the current study. There is no reason to expect that colleges in the southwest differ meaningfully 
from colleges located in other regions of the United States, but it is possible that the results could 
differ if more institutions from other regions were included in the analyses. To mitigate this issue, 
both institutional and student weighting was employed to ensure that the sample was representative 
of the larger population. Finally, the inability to completely differentiate between student transfers 
and dropouts and their effects on academic outcomes in Study 3 was a limitation. Future research 
should address such concerns. 

Given the current national discourse focused on promoting STEM participation, the results suggest 
that caution should be taken against the blanket encouragement of all students to enter a STEM 
field, especially among those who will likely encounter academic hardships given their current level 
of academic preparation in mathematics and science. For example, among the 1 .8 million ACT-tested 
high school graduates in 2014, the percentage of students meeting the ACT College Readiness 
Benchmark in mathematics and science were 43% and 37%, respectively; the percentages drop 
to 16% and 23% for meeting the STEM readiness benchmarks in the same areas. The percentage 
meeting both is even lower: 13%. These results suggest that simply promoting STEM participation 
is not a viable solution to the leaky STEM pipeline. Rather, preparing students academically and 
motivating them to pursue STEM fields as early as elementary and middle school is needed 
(Venkataraman, Riordan, & Olson, 2010). By focusing on earlier grades, efforts can be directed at 
minimizing both the interest and achievement gaps often seen among females and minority students 
in STEM (e.g., OECD, 2014; Venkataraman et al., 2010). Such efforts would hopefully have the 
downstream effects of increasing the percentages of students graduating high school ready for 
STEM coursework to numbers higher than 16% in mathematics and 23% in science. Focusing on 
STEM readiness at the end of high school is too late to remedy the academic deficiencies and/or 
lack of interest (Doughterty, 2014; Mattern et al., 2014). 

In sum, the goal of this research was to develop a metric that would be useful to students to gauge 
their preparedness for STEM-related coursework. Hopefully, for students who are not STEM ready 
but are interested in pursuing a STEM career, this information will compel them to take preemptive 
actions to further develop their knowledge and skills in these areas so that they are in a better 
position to achieve their educational and career aspirations and goals. ■ 
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Appendix 

Table A1. Parameter Estimates from Hierarchical Logistic Regression Models for Course 
Success 


B or higher C or higher 


College 

course 

ACT subject 
test 

Mean/ 

variance 

Intercept 

(SE) 

Slope 

(SE) 

Intercept 

(SE) 

Slope 

(SE) 



Mean 

-4.247 

0.159 

-2.693 

0.138 

Calculus 

Mathematics 

(0.366) 

(0.012) 

(0.355) 

(0.012) 

Variance 

3.188 

(1.191) 

0.003 

(0.001) 

2.951 

(1.029) 

0.003 

(0.001) 



Mean 

-4.527 

0.180 

-2.642 

0.152 

Combined 

Science 

(0.210) 

(0.008) 

(0.202) 

(0.008) 

Science 

Variance 

3.857 

(0.662) 

0.005 

(0.001) 

3.514 

(0.637) 

0.005 

(0.001) 


Note. Mean = estimated mean value of parameter across institutions; variance = estimated variance of parameter across institutions. 
Combined Science includes Biology, Chemistry, Physics, and Engineering courses. SE = standard error. 
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Table A2. Parameter Estimates from Hierarchical Logistic Regression Models for Students’ 
Chances of Earning a 3.0 or Higher Cumulative GPA in STEM 


Subject area 

Year 

Mean/ 

variance 

Intercept 

(SE) 

STEM benchmark 
indicator (SE) 



Mean 

-0.461 

(0.056) 

1.367 

(0.058) 



Variance 

0.124 

(0.030) 

0.062 

(0.027) 


2 

Mean 

-0.098 

(0.054) 

1.262 

(0.058) 

Mathematics 

Variance 

0.106 

(0.028) 

0.031 

(0.018) 

3 

Mean 

0.111 

(0.063) 

1.160 

(0.059) 


Variance 

0.145 

(0.038) 

0.021 

(0.015) 


4 

Mean 

0.265 

(0.068) 

1.103 

(0.065) 


Variance 

0.164 

(0.043) 

0.036 

(0.022) 



Mean 

-0.494 

(0.058) 

1.076 

(0.046) 



Variance 

0.137 

(0.033) 

0.045 

(0.018) 


2 

Mean 

-0.164 

(0.053) 

1.167 

(0.060) 

Science 

Variance 

0.101 

(0.027) 

0.066 

(0.028) 

3 

Mean 

0.049 

(0.063) 

1.069 

(0.053) 


Variance 

0.142 

(0.037) 

0.033 

(0.016) 


4 

Mean 

0.204 

(0.068) 

1.022 

(0.059) 


Variance 

0.165 

(0.044) 

0.040 

(0.021) 


Note: Mean = estimated mean value of parameter across institutions; variance = estimated variance 
of parameter across institutions. SE = standard error. 
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Table A3. Parameter Estimates from Hierarchical Multinomial Regression Models for STEM 
Persistence 





Not enrolled vs. persisted 

Left STEM vs. persisted 

Subject area 

Year 

Mean/ 

variance 

Intercept 

(SE) 

STEM benchmark 
indicator (SE) 

Intercept 

(SE) 

STEM benchmark 
indicator (SE) 



Mean 

-0.583 

-0.846 

-1.020 

-0.814 


2 

(0.076) 

(0.031) 

(0.150) 

(0.030) 


Variance 

0.263 

(0.057) 

— 

1.054 

(0.240) 

— 



Mean 

-0.082 

-0.891 

-0.668 

-0.922 

Mathematics 

3 

(0.084) 

(0.028) 

(0.150) 

(0.029) 

Variance 

0.322 

(0.069) 

— 

1.052 

(0.241) 

— 



Mean 

0.164 

-0.933 

-0.560 

-0.958 


4 

(0.089) 

(0.027) 

(0.153) 

(0.029) 


Variance 

0.368 

(0.078) 

— 

1.086 

(0.250) 

— 



Mean 

-0.569 

-0.658 

-1.011 

-0.618 


2 

(0.080) 

(0.027) 

(0.152) 

(0.027) 


Variance 

0.287 

(0.062) 

— 

1.081 

(0.245) 

— 



Mean 

-0.068 

-0.692 

-0.660 

-0.695 

Science 

3 

(0.088) 

(0.025) 

(0.153) 

(0.026) 

Variance 

0.352 

(0.075) 

— 

1.085 

(0.248) 

— 



Mean 

0.179 

-0.729 

-0.552 

-0.719 


4 

(0.094) 

(0.025) 

(0.155) 

(0.026) 


Variance 

0.403 

(0.085) 

— 

1.123 

(0.258) 

— 


Note: Slopes did not randomly vary across institutions; SE = standard error; mean = estimated mean value of parameter across 
institutions; variance = estimated variance of parameter across institutions. 
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Table A4. Parameter Estimates from Hierarchical Discrete-Time Survival Models for STEM 
Degree Completion 



Mean/ 

Mathematics 

Science 

Variable 

variance 

Estimate (SE) 

Estimate (SE) 

Term 6 

Mean 

Variance 

-5.639 (0.141) 
0.507(0.166) 

-5.644 (0.143) 
0.526 (0.170) 

Term 7 

Mean 

Variance 

-5.217 (0.147) 
0.678 (0.198) 

-5.225 (0.148) 
0.687 (0.198) 

Term 8 

Mean 

Variance 

-2.015(0.125) 

0.714(0.160) 

-2.038(0.129) 
0.755 (0.169) 

Term 9 

Mean 

Variance 

-3.386 (0.124) 
0.583(0.147) 

-3.412(0.129) 
0.632 (0.158) 

Term 10 

Mean 

Variance 

-2.859 (0.117) 
0.540 (0.132) 

-2.891 (0.121) 
0.586(0.142) 

Term 11 

Mean 

Variance 

-4.295(0.143) 

0.496(0.162) 

-4.321 (0.145) 
0.518(0.167) 

Term 12 

Mean 

Variance 

-3.863(0.111) 
0.279 (0.097) 

-3.894 (0.114) 
0.306(0.104) 

STEM Benchmark 
Indicator 

Mean 

Variance 

1 .030 (0.053) 
0.071 (0.023) 

0.836 (0.045) 
0.048 (0.016) 


Note: The term variables can be interpreted as Term 6 = Spring/Summer term of Year 3, Term 7 = Fall term of 
Year 4 through Term 12 = Spring/summer term of Year 6. Mean = estimated mean value of parameter across 
institutions; variance = estimated variance of parameter across institutions. SE = standard error. 
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